Reinforcement Learning in Evolutionary Games∗

We study an evolutionary model in which strategy revision protocols are based on agent specific characteristics rather than wider social characteristics. We assume that agents are primed to play a mixed strategy, with the weights on each pure strategy modifiable on the basis of experience. At any time, the distribution of mixed strategies over agents in a large population is described by a probability measure on the space of mixed strategies. In each round, a pair of randomly chosen agents play a symmetric game, after which they update their mixed strategies using a reinforcement learning rule based on payoff information. The resulting change in the distribution over mixed strategies is described by a non-linear continuity equation — in its simplest form a first order partial differential equation associated with the classical replicator dynamics. We provide a general solution to this equation in terms of solutions to an associated finite-dimensional dynamical system. We use these results to study in detail the evolution of mixed strategies in various classes of symmetric games, and in a simple model of price dispersion. A key finding is that, when agents carry mixed strategies, distributional considerations cannot be subsumed under a classical approach such as the deterministic replicator dynamics.